Field of the Invention
The invention relates to a method for calibrating a camera system of a motor vehicle as well as to a corresponding apparatus.
Description of the Background Art
Today's motor vehicles having driver assistance systems are equipped with a large number of sensors, many driver assistance systems depending on the data of cameras. To ensure a reliable operation of the driver assistance system, these cameras must be calibrated. According to the prior art, a complex, static calibration is carried out in the factory for this purpose. To eliminate the static calibration, a robust online calibration system is needed, which provides suitable calibration results after a short driving distance, such as the three orientation angles of the camera and its height above the ground.
In M. Siebeneicher: “Eine automatische Kalibrierung beliebiger Kameras in Fahrzeugen auf Grundlage von optischem Fluss and Fluchtpunktbestimmung” (Automatic Calibration of an Arbitrary Camera in Vehicles Based on Optical Flow and Vanishing Point Determination), Master's Thesis, Freie Universität Berlin 2008, a method is explained, which determines the orientation angle of the camera by evaluating the optical flow based on the movement of concise feature points and ascertaining the vanishing point of the optical flow. To ascertain the vanishing point, the movement vectors of the feature points are intersected with each other, wherein the points of intersection should theoretically coincide with the vanishing point, which, however, is only approximately the case, due to measuring inaccuracies, so that the maximum of all points of intersection are taken as the vanishing point. To reduce measuring inaccuracies, the points of intersection are followed over time with the aid of a particle filter, and the vanishing points resulting therefrom image-wise are stabilized with the aid of a RANSAC filtering.
The time profile of the vanishing point is evaluated to determine the roll angle from the horizon. Once a certain number of vanishing points in left and right turn maneuvers have been ascertained, the horizon is determined from a subset of all vanishing points calculated up to that time by analyzing the main components. By comparing the velocity changes and the yaw angle, it is ascertained whether the vehicle is moving uniformly in a straight trajectory, and the pitch and yaw angles may therefore be determined, or whether a turn maneuver is being carried out. In this method, the height is determined separately. However, the system is computationally very complex, in particular due to the determination of features and the complex filtering of the vanishing point.
The method presented in J. Platonov et al: “Vollautomatische Kamera-zu-Fahrzeug-Kalibrierung” (Fully Automatic Camera-to-Vehicle Calibration), ATZ elektronik 2012, pp. 120-123, describes a technique for the purely image-based determination of the camera orientation without determining the camera height. The method is based on the technique of visual movement estimation. The first method determines the vehicle movement between two adjacent images, and a second method examines the point correspondence between six images, the second method being more accurate but also much more computationally complex. To determine the orientation angle, the camera movement is divided into the classes of “straight movement” and “maneuver.” The pitch and yaw angles are estimated during the straight movement and the roll angle during the maneuver.
A method is presented in E. Tapia et al: “A Note on Calibration of Video Cameras for Autonomous Vehicles with Optical Flow,” Department of Mathematics and Computer Science, Serie B Informatik (Series B Computer Science), Freie Universität Berlin, February 2013, which determines the orientation of the camera only on the basis of camera images and determines the camera height with the aid of the vehicle velocity. The optical flow is determined in a manner which is not described in greater detail, and it is demonstrated that the flow vectors intersect at the vanishing point when driving in a straight trajectory. The ground plane is ascertained to determine the horizon, the three Euler angles being able to be determined with the aid of the vanishing point and the horizon. To determine the camera height, the vehicle travels forward, the camera orientation being known, and the camera being directed onto the road so that the road is displayed in the camera image. The start and end points of a flow vector in the image are projected back into the world coordinate system. The camera height may be calculated with the aid of this information, taking into account the vehicle velocity as well as the driving duration.
Publication EP 2 131 598 A2 relates to a stereo camera system as well as a method for ascertaining at least one calibration error of a stereo camera system. With the aid of at least two individual cameras of the stereo camera system, an image sequence of images having depictions of a detection area in front of a vehicle is recorded during a travel of a vehicle along a road. Corresponding image data is generated from the images of the image sequences. The generated image data is processed, the course of at least one edge of the road being ascertained. At least one calibration error is ascertained on the basis of the ascertained course of the at least one edge of the road.
To determine the position of all three angles of the camera orientation as well as the height of the camera above ground, the known systems are computationally intensive and often insufficiently accurate, in particular in poor weather conditions as well as over short driving distances having few turn maneuvers.